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The Probability Inquiry Environment (PIE) is 
being designed as a computer-mediated 
collaborative inquiry environment to aid middle 
school students in learning elementary probability. 
This paper will report on a study in which seventh 
grade students engaged in probabilistic reasoning 
while interacting with a preliminary version of PIE. 
By analyzing the reasoning used by students, it 
was found that the findings from the standard 
“misconceptions” literature do not do justice to the 
wide range of viewpoints voiced by the students. 
In particular, the students did not consistently 
invoke such well documented misconceptions as 
representativeness and the law of small numbers. 
Instead, the students invoked a great variety of 
intuitions, some of which approach normative 
reasoning in probability, and others of which 
interfere with normative reasoning. This paper will 
then discuss how probability instruction can be 
improved by introducing students to a progression 
of inquiry activities that build from the students’ 
existing intuitions (Vahey, 1996; Vahey, Enyedy, 
and Gifford, in press; White 1993a, 1993b). 

Prior Findings in Probabilistic Reasoning 

There is a rich literature based on the many 
misconceptions people display when asked to 
reason probabilistically 1 . By far the most 
influential work has been by Tversky and 
Kahneman (1982), who showed that much of 
people's probabilistic reasoning could be described 
by the heuristics of representativeness and 
availability. The representativeness heuristic is 
characterized by making judgments based on the 
degree to which A is representative of, or 
resembles, B (Tversky and Kahneman, 1982). 



1 Although many terms have been used to denote student 
ideas in formal domains, such as naive conceptions , 
preconceptions, and alternative frameworks, to name a few, 
much of the literature cited here is consistent with a view of 
misconceptions in which student ideas are determined to be 
in conflict with accepted meanings in formal domains (cf. 
Confrey, 1990; Smith et al„ 1993). 



This representativeness heuristic has been used to 
explain insensitivity to sample size, the gambler’s 
fallacy, the base-rate fallacy, incorrect judgments 
about the output of random processes, and other 
non-normative judgments. The availability heuristic 
is characterized by making judgments based on the 
ease with which instances of a certain event can be 
brought to mind. This heuristic has been used to 
explain biases due to the retrievability of instances 
(such as thinking that car accidents are more likely 
in a town because you had a car accident in that 
town), biases due to the effectiveness of a search 
set (for instance, claiming that there are more 
words that end in “ing” than have an “n” as the 
second to last letter), and biases of imaginability 
(things that are hard to construct a mental image of 
are determined to be not likely). 

In addition to those found by Kahneman and 
Tversky, many other misconceptions have been 
identified (for an overview, see Shaughnessy, 
1992). For example, people may believe that there 
is a lack of variability in the world, people have too 
much confidence in small samples, people do not 
see the importance of small differences in large 
samples, and people seem unaware of regression to 
the mean in their lives (Shaughnessy, 1992). 

Although this misconceptions literature has been 
fruitful in identifying some of the problems people 
have in reasoning probabilistically, several 
objections have been made to the misconceptions 
literature. General attacks point out that much of 
the misconceptions literature is based on the 
paradigm of expecting students to “replace” their 
current knowledge after being “confronted” with 
counter-evidence to their “misconceptions”. This 
is an overly simplistic account of learning, and 
misconceptions studies do not provide an account 
of ways people can change their intuitions (which 
are often plagued by misconceptions) into more 
expert knowledge (Smith, et al., 1993), although 
such an account would be perhaps the single most 
important factor in designing learning 
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environments. Specific attacks on much of the 
literature in probabilistic reasoning point out that 
much of the misconceptions literature is based on 
questionnaire and survey data, providing no 
account of the actual thought processes people use 
(cf. Konold et al., 1993; Shaughnessy, 1992), 
leaving researchers to infer people’s reasoning. 
Additionally, there is no explanation for the 
variability of subjects’ responses — for instance, 
there is no explanation as to why some subjects use 
representativeness, but others do not (Tversky and 
Kahneman, 1982). Finally, documenting such 
misconceptions in counterintuitive situations does 
not provide insight into correct reasoning people 
may employ when reasoning probabilistically 
(Hawkins and Kapadia, 1988; Lajoie, 1995). 

Consistent with many of these objections, this 
study found that students did not consistently fall 
prey to misconceptions such as representativeness 
or the law of small numbers, although there was 
evidence that, for some students in some 
situations, such misconceptions did describe their 
behavior. Instead there was great variability as to 
the justifications and explanations provided, and 
this variability was found both across groups of 
students and within individual students. In short, 
if we are to design learning environments that are 
informed by our understanding of student 
conceptions of probability, we need to achieve a 
better understanding of student conceptions. 

The goals of this paper are to advance our 
understanding of middle school students’ 
probabilistic knowledge by analyzing seventh 
grade students’ reasoning as they used the 
Probability Inquiry Environment; determine the 
ways in which such reasoning can be mapped to 
normative reasoning in probability; and begin to 
investigate how we can use this mapping to create 
authentic and collaborative learning environments 
that exploit students’ existing conceptions as 
resources for cognitive growth. 

Theoretical Framework 

This study is situated in a constructivist view of 
learning that is based upon three areas of research: 

(i) existing student conceptions play a 
productive role in the development of 
expertise, and we should not view 
misconceptions as deficiencies that must be 
replaced, but instead we should work to build 
from these conceptions (for a thorough 
treatment of this perspective, see Smith, et al., 



1993; see also diSessa, 1988; Minstrell, 1989; 
Lampert, 1986; White, 1993a, 1993b) 

(ii) students construct meaning and 
understanding while participating in a social 
context (Brown and Campione, 1996; Collins 
et al. 1989; Scardemalia and Bereiter, 1993) 
and student learning is facilitated by making 
their existing ideas explicit in order to evaluate 
these ideas with respect to findings in the 
domain and the ideas of others 

(iii) learning environments should situate 
students in activities they consider to be 
legitimate and authentic. A productive learning 
environment will support activities that 
maintain fidelity with the relevant aspects of 
authentic practice, and not ignore the 
contribution of social practices and 
representational resources. That is, we should 
design intermediate tools and models that 
maintain fidelity with the relevant aspects of 
expert analysis while students participate in 
authentic tasks (Gordin et al. 1994; White, 
1993a, 1993b) 

The Environment 

The Probability Inquiry Environment (PIE) is 
being created as a collaborative, guided-inquiry 
learning environment in which seventh grade 
students are asked to evaluate the fairness of games 
of chance. PIE was orignially based on a 
provisional set of conjectures that was informed by 
the existing literature on students’ conceptions in 
probability. These conjectures led to PIE 
consisting of a set of inquiry activities designed to 
address specific “misconceptions” in probability, 
such as representativeness and the law of small 
numbers. Because of the prototype nature of this 
version of PIE, and because of the short time the 
students interacted with PIE in this study, we did 
not expect to make a significant gain in terms of 
students’ understanding of probability during this 
study. Instead, because PIE was designed as an 
open-ended collaborative inquiry environment, the 
purpose of this study was to discover the specific 
probabilistic intuitions that students included in 
their explanations and discussions 2 . 



2 It should be noted that another purpose of this study was 
to test the interface of PIE with seventh grade students, and 
not all students understood all aspects of the interface. 
Several of the students interpreted specific questions in ways 
that were unexpected, and so there is no in-depth analysis of 
responses to specific questions. Instead student justifications 
are analyzed (for whatever question they thought they were 
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We chose analyzing the fairness of games of 
chance because we have found that students are 
interested in the notion of fairness as it applies to 
games of chance (Vahey, 1996). Additionally, the 
literature on moral development shows that middle 
school children are interested in notions of 
fairness, and students of this age have developed 
sophisticated notions of fairness (Damon, 1988; 
Thorkildsen, 1995). 



In this study students were asked to analyze two 
games of chance to determine if they were fair. 
The first game is called the Two-Penny game, 
where Team A scored a point whenever both coins 
come up the same (heads-heads or tails-tails), and 
Team B scored a point whenever both coins come 
up differently (heads-tails or tails-heads). This 
game is fair, as all outcomes are equally likely, and 
each team scores on two out of the four possible 
outcomes. The second game is called the Three- 
Coin game, where Team A scored a point on five 
of the eight possible outcomes, and Team B scored 
a point for three of the possible outcomes. 
Because each outcome is equally likely and Team A 
scores on more outcomes than Team B, this game 
is unfair in favor of Team A. An event tree that 
enumerated all the possible outcomes and that 
visually presented the scoring combinations for 
each team was on the screen at all times, as was a 
dynamic histogram that showed scoring either by 
each combination of coins, or by each team (see 
figures la and lb). 



O I cmw iw w tur 

# * Ik* f**4 M Mrf «| tt 

O i flwt 

So. 

liJroi ^ 




T«olVf way Gome 

i \ s? MNM " 





J ,v>: 






Two Penny Game 

Figure la: the PIE software interface 



answering) and the probabilistic reasoning used in these 
justifications is analyzed. 




Three Coin Game 

Figure lb: the PIE software interface 



An inquiry approach was chosen because 
cognitive science research in the field of 
mathematics has demonstrated that the inquiry 
approach to mathematics instruction can provide a 
rich, engaging and meaningful context which 
facilitates students’ acquisition of mathematical 
concepts (Balachef, 1987, cited in Schoenfeld, 
1991; Lampert, 1995; Richards, 1991), and 
investigating complex concepts in this manner 
scaffolds the development of metacognitive 
knowledge and skills (White & Frederiksen, 
1995). In PIE students collaborate in the authentic 
tasks of: creating conjectures to explain genuinely 
puzzling situations; running a computer simulation 
that generates data that can be used to test their 
conjectures; analyzing this data to determine the 
appropriateness of their conjectures; and creating 
and negotiating an understanding that accounts for 
their findings, which is then agreed upon and typed 
into the computer environment. During this inquiry 
cycle students use authentic representations and 
tools such as event trees, simulations, and real-time 
graphs and histograms. 

The inquiry cycle used in this version of PIE 
consisted of five steps: Rules, Try, Predict, Play, 
and Conclude. Throughout each of these five steps 
the right side of the screen remained relatively 
constant, and always showed a probability tree that 
denoted all the possible outcomes of the coins 
flips, with each outcome labeled according to 
which team, A or B, scored for that outcome. That 
is, the partitioning of all outcomes remained on the 
screen at all times. In Rules the students were 
shown an animated introduction to the current 
game. In this introduction the students were shown 



Paper presented at the 1997 AERA Conference 




9 Phil Vahey 





that the coins are flipped, and the outcomes that 
score a point for each team were explicitly shown 
to the students. In Try the students were 
introduced to the coins, probability tree, histogram, 
and controls by allowing them to manipulate these 
items by clicking them with the mouse. This was 
done to allow the students some amount of 
familiarity with the environment before asking 
them to make predictions. In Predict and 
Conclude the students were asked to make 
predictions and conclusions, respectively, and 
these predictions will be discussed in more detail in 
the following paragraphs. In Play the students 
started and stopped the game by clicking on a 
button that toggled between “Go” and “Stop”, were 
able to control the speed of the coin flips with the 
fastest speed flipping the coins at a rate of about 
100 points a minute, could switch the histogram 
between viewing by combination or viewing by 
teams, and had a set of hints that changed based on 
the status of the game (recommending, for 
instance, that students play at the fastest speed if 
after many points they had not already done so). 
As the coins flipped in Play the probability tree 
highlighted the current state of the coins. 

The Prediction and Conclusion questions were 
similar across both activities (the two coin game 
and the three coin game). There were three sets of 
prediction questions (figure 2). The first set of 
questions asked students about fairness: was this 
game fair, what do they mean by fair, and why is 
this game fair or unfair. The second set of 
questions had students manipulate histograms to 
make predictions about what they might expect 
after 10 points and after 200 points. The third set 
of prediction questions asked students to predict 
the relative frequency of each combination. The 
first three conclusion questions paralleled the 
prediction questions — students were shown their 
predictions and were allowed to keep their 
predictions as conclusions, or enter new 
conclusions. One additional set of conclusion 
questions was asked in which students specified 
the most important thing in deciding the fairness of 
this particular game, and were explicitly asked if 
the number of combinations that scored for teach 
team were important. 




Eto you think that this is a fair game? 

I think the game is fair. 

I think the game is not fair. 

I dcn't know. 

Hew marry points do you predict that Tean A ard Tean B 
will have scored after playing for 10 turns? After 
200 turns? Shew how marry points you think each team 
will score by pulling up and down the bars above. 

Do you think any carfcdnaticn of coins is going to happen 
more often than ary other? Pull each Yellow bar to 
determine hew often you think that carbinaticn of 
coins will occur. 

Figure 2: Prediction Questions 

These games and questions were designed to 
elicit specific misconceptions. For example, we 
expected that the first game would address some 
aspects of the representativeness misconception. 
Because Team B scored a point whenever the coins 
where “mixed”, we expected most students to 
answer that this game was unfair, in favor of Team 
B. We expected that the three different prediction 
questions would allow the students to look at the 
same situation from three different perspectives, 
providing a way for them to integrate their 
understanding of different aspects of probability. 
This expectation was too simplistic: asking what 
we considered to be similar questions from several 
perspectives did not provide an opportunity for 
students to integrate their knowledge, but instead 
provided students with an opportunity to invoke 
different reasoning strategies. 

The Study 

The research team recruited eight single-sex pairs 
of students from a local urban middle school. All 
of the sessions took place in the summer, between 
the students’ sixth grade and seventh grade school 
years. Four of the pairs were boys, and four of the 
pairs were girls. The students represented a wide 
range of ethnicities. The students were paid $5 and 
hour, and spent about an hour and a half interacting 
with the computer during one two hour session. 
During this session they were videotaped, and PIE 
recorded their actions on the computer. 

The data used in this study consists of student 
discussions as they participated in the activities, 
and their responses to the predict and conclude 
questions. The videotape data of the students 
using PEE were transcribed, and these transcripts 
were combined with the data recorded by PIE to 
create a record of all student discussions and 
student interactions with PIE during the session. 
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These transcripts were then analyzed, and all 
instances of students’ reasoning about the games 
was found. Although, by the end of the study, 
seven of the eight pairs of students were able to 
reason normatively about the games, this paper will 
not concentrate on the events that led to this 
normative reasoning. Instead, the students’ 
reasoning throughout the entire session will be 
compared to the way in which normative 
probabilistic reasoning could have been employed 
in those situations. The characteristics of 
normative probabilistic reasoning discussed in this 
paper corresponds to four interrelated conceptual 
areas in probability. These areas are randomness 
(understanding that the game is based on a non- 
deterministic mechanism), the outcome space (the 
possible outcomes for each point played), the 
probability distribution (the probability of each 
outcome), and the validity of evidence (the law of 
small/large numbers). A justification for this 
characterization will be presented after the next 
section. First, to acquaint ourselves with an 
example of student reasoning in PIE, a brief case 
study will be presented. 

A Case Study of Q and T Using PIE 

When Q and T were assured that they understood 
the two-penny game (Figure 3), they made their 
predictions. When predicting that the Two-Penny 
game is fair, they explicitly assigned a 50% chance 
to the combinations of coins that score a point for 
each team, resulting in a final answer that is 
perfectly aligned with normative reasoning (Figure 
4). Note however, that the students never explicitly 
justified this 50% chance, and we will not attempt 
to make claims about the students’ probabilistic 
reasoning in this instance. 

T: I get it, if they're both heads team A gets a point, 
if they're both tails team A gets a point, and if 
they're cne heads and one tails Team B gets a point. 

Figure 3: Understanding the two-penny game. 



fairness. Student ideas of fairness will be further 
discussed in the next section. 

In the next question, Q and T were asked to 
manipulate histograms to make predictions about 
what would happen after 10 points, and after 200 
points. T stated that heads would occur more, so 
she would expect Team A to win more. Q 
countered this by saying that coins usually come up 
differently (note that this is consistent with 
representativeness), so she would expect Team B 
to win more. They then decided that the game will 
most probably be tied (Figure 5). After further 
discussion, the students decided that luck would be 
an important factor in the game (Figure 6), and this 
meant that Team A might win sometimes, and 
Team B might win other times. Again, the 
students provided answers that were close to 
normative, but looking solely at their final answer 
misses their reasoning process, which was quite 
rich and invoked more intuitions than are found in 
the final answer. 

T: CK, what about this, I think Team A will win. 
Because imagine all of the heads we are gcnna get 

Q: I think that Team B would win it, because vhai you 
threw it it's like real luck vhaa you get both of than 
the same. Vhen you throw it, most of the tine they 
land differently, [pause] I think they'd probably be 
tied, but if I had to choose one, see, this looks 
right to me [pointing to evm histogram bars] 

T: that's what I think too 

Figure 5: Who will win more? 



T: I think that this is just a gams of luck. . .this is 
like a game of guessing 

Q: it's really like saneane has to win, b ecau se it's 
like yoa win sore you lose seme, it's not like a 
permanait game. 

[T moves histograms so A is winning after 10, and B is 
winning after 200] 

Q: new hew can purple [Teem A] be winnirg cn this cne 
and greoi [Team B] be winning cn that? 

T: well, cn ce you win you den' t always win 



Q: we think the game is fair because you have a 50% 
chance of getting both heads ard. both tails. 

Typed: We think that the game is fair because you have 

a 50% chance of getting both heeds and both tails. 
You also have a 50% chance of getting cne tails ard 
cne heads. 












Figure 4: Fair, based on a 50% chance for each team. 

Q and T next answered what they meant by fair. 
Q and T decided that a game is fair if all teams have 
an equal chance of winning, a normative view of 



SLf 



For the final prediction question in the two-penn 
game, the students were asked if any of th 
combinations (outcomes) would happen more tha 
any other. The students were again asked t 
manipulate a series of histograms, and type in 
justification. Although Q and T started to reaso 
about the different combinations, they the 
switched to talking about the probability of head 
or tails. They decided that because this is not 
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Figure 6: A game of luck 



game of skill, but a game of luck, they would 
expect heads and tails to come up the same amount 
(Figure 7). 

Q: we dcn't think that any cmbinatJ.cn should happen 
more than any other, because it's luck 

T: it's hew you throw it 

Q: plus, it's a gane of luck 

T: we don't think that, it's not a game of skill 

Q: that one penny will core up more than the other 

T: it's all equal... wait a minute, we don't think that 
heads or tails is more likely to oerre up than the 
other. We dcn't think that heads or tails will oerre 
up more 

typed: We dcn't think that heads would oerre up more than 
tails. We think that this gane is a game of luck. 

Figure 7: Will any combination happen more? 

Q and T then started the game simulation. After 
only four points they noted that tail-head was 
happening more than the other combinations. The 
game then stopped after ten points, and told the 
students that they could either look at the results or 
continue playing. At this point Q wanted to go 
back into predictions to see if the predictions 
agreed with the results (note that this behavior is 
consistent with the well-documented law of small 
numbers). The researcher asked them to continue 
playing, telling them that they would be able to 
modify their predictions later, so Q and T 
continued in Play. 

After playing for several more points, Q and T 
then decided to run the simulation at the fastest 
speed. At this speed the game runs ten points at a 
time, and individual coin flips can not be 
perceived. After about 20 seconds the game 
reached 200 points, and gave the students the 
option to continue playing or stop and analyze the 
results. Q and T chose to stop playing and go 
immediately into Conclude. 

The first three conclusion screens asked the 
students to evaluate their predictions. For each of 
these conclusions, the students stated that, 
although their predictions did not exactly match the 
actual data, the results were close enough for them 
to still agree with their predictions (Figure 8). 
Additionally, in Figure 9, Q came back to their 
earlier statement that being a game of luck is an 
important aspect of the game. This becomes the 
single most important factor for these students for 
the remainder of the session. 




T: even, look, almost even 

Q: yeah, so it's pretty fair because no one is like way, 
way more than the other 

Figure 8: Conclusion — is the game fair? 



Q: No, CK, why? We keep saying the sane thing over. 
Like this evidence-no because this is just a gate of 
luck. And they're all equal anyway. 

Figure 9: Conclusion — are any combinations more likely? 

For the last set of conclusions the students were 
asked if the number of combinations that score a 
point for each team is an important factor in 
determining fairness. However, Q and T 
understood this question to be asking about the 
data already collected. Although they first stated 
that this data is important, they then decided that 
luck is more important, and one doesn’t need to 
analyze the data (Figure 10). Then, when asked to 
state the most important thing in determining if the 
game is fair, they again stated the importance of 
luck (Figure 11). 

Q: very inportant, dcn't you think. . .why is it very 
inportant? 

T: it's data 

Q: it's inportant data, and urn 

T: you need the data to play the game 

Q: ytu need to know the nurrber of oerrbinatiens . . .But 
it's not that inportant though, as a natter of fact, 
it's not inportant at all, cause it's a game of luck. 
Yeah, it's not inportant 

T: it's a little inportant. . . 

Q: . ...hit the gane is just a gane of luck anyway. So if 
you didn't have the data, it wouldn't natter anyway. 

Typed: It is inportant because it's data, but an the 
other hand it is not that inportant because it's just 
a game of luck. 

Figure 10: It’s just a game of luck 



Q: The most inportant thing is that you urderstand that 
the game is just luck 

Figure 11: The most important thing in determining 
fairness 

Q and T were then introduced to the Three-Coin 
game, and Q once more decided that the game was 
fair because it was a game of luck, although T was 
hesitant to agree. However, T could not state why 
she thought the game was unfair, and finally 
determined that, since all the outcomes were 
possible, the game must be fair. Note that, 
although the partitioning of the outcomes into 
points for A and points for B were on the screen at 
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all times, the students never considered counting 
the outcomes to determine if each team had an 
equal number of outcomes. Although it is 
dangerous to make inferences based on the absence 
of an action, the fact that it never occurred to these 
(or most) students to simply count up the number 
of outcomes, especially when something about the 
game seemed troubling, may point to a lack of an 
understanding of the importance of the outcome 
space in determining probabilities. 

Then, consistent with their predictions for the 
Two-Penny game, and consistent with their idea 
that luck means that a game is fair, for the 
remainder of the predictions Q and T stated that the 
teams will score an approximately equal number of 
points, and each of the combinations should occur 
equally. They then put the game on the fastest 
speed, and quickly played up to two hundred 
points. When the game reached two hundred 
points, they went into conclude and simply agreed 
with all their predictions, without comparing their 
predictions to the actual results, even stating that 
the data is not important (Figure 12). Such 
reasoning is based, presumably, on the statements 
made at the end of the Two-Penny game, that a 
game of luck must be fair, and data is not an 
important factor. 

Typed: THE COINS IS JUST DATA AND THE DATA I N THIS GAME 

IS NJT THAT IMPCREANT. 

Figure 12: Conclusion — are the combinations in the 
Three-Coin game important? 

At this point the researcher asked them to play 
some more, reminding them of the Reset button 
that sets the points back to zero. After playing 
several more rounds up to two hundred points, T 
decided that, because Team B kept losing, the 
game must be unfair. Q, however, kept stating that 
the game is just luck, and so must be fair (Figure 

13) (note the contrast between this behavior and 
her earlier desire to check their predictions after 
only ten points). T did not accept this answer, 
however, and kept looking for an explanation. T 
finally noticed the difference in the number of 
outcomes that scored a point for each team, and 
then resorted to a strategy of making the game fair, 
contrasting a partitioning of points that would make 
the game fair with the actual partitioning (Figure 

14) . Q then understood how this partitioning was 
relevant and agreed that the game was unfair. 




T: See, Tean B is losing by a lot. Tbld you it was 
unfair 

Q: this game is just luck, it's just a pamy game, it's 
just luck 

Figure 13: Unfair versus luck 

T: vhy is it fair, Q? 

Q: because it's a game of luck, it's jut throwing 
pennies 

T: I'm not talking about whose got tie pamy, I'm 

talking about right here [pointing to tie tree] . They 
keep losing, and I'm trying to figure out vhy. . .wait a 
minute! See hew this is AA right here? and this is AB 
AB AB 

R: nrrm-hnrm 

T: but shouldn't it be BB? 

R: what do you mean, shouldn't it be? 

T: right here it says AA AB AB AB 

Figure 14: Making the game fair 

Summary: Q and T’s analysis of the Two- 
Penny game began with them stating several 
different, often competing or conflicting, intuitions 
about probability, few of which seemed to carry 
any deep commitment. And, although 
representativeness could be used to describe some 
of their reasoning, it is a far from adequate 
account, as much of their reasoning is inconsistent 
with representativeness. Q and T then began to 
consider luck the single most important aspect of 
the game, even stating that they did not need data to 
determine the fairness of the games. Their 
commitment to this position was shown in the 
Three-Coin game, when Q explicitly denied the 
importance of data that showed that this game was 
unfair. Note that she did not fall prey to the law of 
small numbers, nor did she dismiss the game as 
“cheating”, nor did she suffer from confirmation 
bias and misinterpret the data as showing that the 
games were tied. Instead, she acknowledged the 
results of the simulation and simply said that these 
results were not relevant. It was not until T was 
able to determine that the partitioning of the 
outcome space was unequal that they were able to 
confirm that the game was unfair, and it took a 
notably long time until this counting strategy was 
employed by the students, lending credence to the 
supposition that the outcome space was not a 
salient feature of this situation. Note that most of 
this behavior is not consistent with the heuristics 
and biases view of probabilistic reasoning. It 
seems as though we need another view of students’ 
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conceptions of probability that is different from that 
offered by the traditional literature. 

Results: a framework for understanding 
probabilistic reasoning 

As mentioned earlier, PIE was specifically 
designed to elicit students’ ideas about 
representativeness and the law of small numbers. 
However, as the case study of Q and T shows, 
such a simple explanation does not account for 
much of the reasoning employed by the students. 
In the spirit of the constructivist approach outlined 
earlier, we must analyze this reasoning to 
determine the ways in which student intuitions can 
be expanded to include the concepts in the domain 
of probability if we are to design instruction that 
helps students bridge from their existing 
knowledge to the formal domain knowledge of 
probability. This section begins to do this by 
analyzing the probabilistic statements of all the 
students in this study. 

The primary activity the students were engaged in 
was determining if the two games of chance were 
fair. This activity, situated in PIE, provided the 
students with an environment where they could 
investigate an activity that was, to them, authentic 
and meaningful, and all pairs of students invoked 
reasoning that is easily recognized as probabilistic. 
In order to understand how this intuitive reasoning 
compares to normative reasoning, we must first 
have an understanding of what we mean by 
normative reasoning in this activity. The version 
of “normative” reasoning presented here is that of 
the idealized reasoning process used by someone 
with an understanding of elementary probability 
who is faced with a novel situation. The novel 
situation in this case is determining if the games of 
chance described previously are fair. 

This normative reasoning will first determine 
what is meant by “fair”. After this any of the 
several different reasoning processes considered 
“normative” will have the characteristics that they 
will (i) determine that the game is based on a non- 
deterministic mechanism (i.e. understand some 
aspects of randomness); (ii) determine the 
outcomes that score points for each team (i.e. 
understand the outcome space); (iii) determine the 
probabilities of the outcomes that score a point for 
each team (i.e. understand the probability 
distribution); and (iv) compare the expected 
fairness of the games to the actual fairness after 
playing for some large number of points to 



determine if the theoretical expectations are 
accurate 3 . 

The main finding is that students displayed a 
wide variety of ideas, some of which approach 
normative reasoning in probability, and others of 
which interfere with normative reasoning, and 
there is great difficulty in attempting to characterize 
students as exhibiting specific misconceptions. 
Many of the students’ statements were situation 
specific, and students were willing to make 
contradictory claims when discussing the same 
basic circumstance, such as the expected results 
after many flips of a coin. Instead of attempting to 
characterize each student as believing some specific 
misconception, it is perhaps more instructive to 
consider ways in which we can elicit their different 
ideas, and attempt to build from those ideas that 
seem most productive. 

Fairness 

Several studies have determined that fairness is a 
productive study of inquiry for students 
investigating probability (Hirst, 1977; Scheinok, 
1988), and PIE is based on this premise. The 
findings from this study will show that, although 
fairness is a highly motivating and productive area 
of investigation for middle school students, we 
should be aware of different versions of fairness 
that students consider relevant in different 
situations. 



3 Note that this description of normative reasoning can 
remain agnostic with respect to specific beliefs about the 
probability of the elementary events involved. For instance, 
one reasoning strategy that fits this description of normative 
is as follows. The students believe that heads occurs 
systematically more than tails. The students then notice all 
the outcomes that score a point for the different teams, and 
then determine how the increased probability of heads 
compared to tails changes the probability of each outcome, 
and hence, the probability of scoring for each team (as an 
aside, the increased probability of heads-heads more than 
offsets the increased probability of one head and one tail, 
making this game unfair in favor of Team A in this 
scenario). The students then run a few simulations of the 
game and notice that, at a few hundred points, the teams are 
only a few points apart, and no team systematically wins 
more than the other team. At this point the students should 
either not believe the computer simulation, or revisit the 
assumptions of their theory. There are obviously many 
other reasoning strategies that could be considered in this 
discussion, but the point here is that any such reasoning 
strategy will take into account characteristics (i) through (iv) 
mentioned previously. 
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Each pair of students discussed the normative 
conception of fairness at least once during the 
session (see figure 15), stating that for a game to 
be fair all teams should be equally likely to win. 
Although Metz (1995) has found that younger 
students may understand fair to mean that the teams 
must score in lock-step fashion, students in this 
study exhibited no such ideas, and some explicitly 
stated that fairness means that there may be 
variation in outcomes, but that variation should not 
systematically favor one team (figure 16). 



UV: {typed) : FAIR IS IF THE TEAMS ALL HAVE A EVEN 

QffiNCE OF WIMHN3. 

0: It's not easier for one player to win then the 
others. 

T: or you could say it would be fair if both teams got 
the same chance. 

Figure 15: normative ideas of fairness 



S: yeah, so it's pretty fair because no one is like way, 
way more than the other 



K: I think it might be fair, because sore times one is 
losing and cne is winning 

Figure 16: fair means that there will be variation 

These were not the only notion of fairness 
invoked by the students. Two pairs of students 
stated that a game was fair if no one cheated (figure 
17). Other students stated that a game was fair if it 
was possible to win (figure 18). Although such 
reasoning could be used to pronounce the Three- 
Coin game (an unfair game) as a fair game, none of 
the students in this study did so. However, some 
students who predicted that the Three-Coin game 
would be fair, after running the simulation and 
seeing that Team A won almost all of the games, 
did use different versions of fairness to justify why 
their predictions may still be correct. Two students 
stated that a game could be fair for one team, but 
not fair for the other. That is, if a team is winning, 
they could perceive the game as fair, whereas a 
losing team could perceive the game as unfair 
(figure 19). Although some of these ideas allowed 
students to maintain that the Three-Coin game was 
fair, even in the face of contrary evidence, students 
tended to hold more strongly to the normative 
notion of fairness, which states that the expected 
outcome should be equal for both teams. That is, 
students were typically willing to give up on non- 
normative notions of fairness after they saw that 




the Three-Coin game was unfair because Team A 
had more outcomes than Team B. 



EN (typed) : "BECAUSE ITS NOT A CHEATING GAME. - 



S: yeah, equal, no cheating, no cne has a better ability 
of winning, it's up to you to win. 

T: no cne can cheat. 

Figure 17: why is this game fair (part 1: no cheating) 

L: yeah, it's not like inpossible to win 

V: [it is fair if] if every tean has a chance 

O: I think it's fair because everybody has a chance. 

Figure 18: why is this game fair (part 2: everyone has a 
chance) 

M: [the game is fair] because it lets ire win. 

C: I think it's fair because I'm winnire? 

Figure 19: Game is fair for one team 

Given this variability in notions of fairness, it is 
important that we ask how can we use fairness to 
help students learn probability. The answer 
offered here is that the context of fairness of these 
games was engaging and motivating to these 
students, and was a productive method of getting 
them to talk about probability theories in a context 
they considered authentic. There is caveat in that, 
although most students may offer normative 
versions of fairness in some contexts, this does not 
mean that students will continue to use those ideas 
of fairness in other contexts. Fairness is an issue 
that should be discussed and revisited throughout 
any unit of probability that depends upon fairness, 
with the different definitions of fairness compared 
and contrasted. Because most students do embrace 
the normative definition of fairness, it seems as 
though we can be optimistic in terms of expecting 
students to expand this definition of fairness to 
apply to most situations involving probabilistic 
reasoning. 

Randomness: 

During the course of the study, every pair of 
students made reference to the fact that non- 
determinism was an important factor in analyzing 
the outcomes of coin flips. This reference typically 
came through students talking about the game 
being based on “luck” or “chance”, and also by 
contrasting these games with games of skill. 
Additionally, students stated that the random 
process of coin flips would result in variation 
between trials (figure 20). Expecting variation 
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from a random process is consistent with the 
normative view of randomness. 



K: It won't be totally even 

S: it's really like sanecne has to win, because it's 
like ycxi win same you lose sane, it's not really like 
a permanent game. 

Q: the tides can change 

Figure 20: Randomness has variability 

However, as seen in the case study of Q and T, 
some students also stated a conception about 
randomness that led to their believing that the 
Three-Coin game was a fair game. Three of the 
eight pairs of students stated that “luck” or 
randomness meant that nothing could be predicted 
about the games. When these students 1 discussed 
the game being based on luck, they were stating 
that no predictions could be made about how the 
flips would turn out, and since anything could 
happen, the games must be fair (see figure 21). 
This result may be consistent with the “outcome 
approach” as described by Konold (1991, Konold 
et al. 1993), as the students were replying to a 
question about a series of events as if the answer 
depended upon being able to predict any given 
event. As mentioned in the previous section on 
fairness, and as will be discussed in the later 
section on data validity, these pairs of students did 
not accept that the Three-Coin game was unfair 
until they noticed the difference in the number of 
paths, and then were able to create a new 
understanding that could explain the data. 



KL (typed) : I THINK THIS GftME IS FTOR BECAUSE IT'S JUST 
LUSK CR F70E VMAT IT LANDS CN! !•!!!! 

K: because it's luck is the answer for the whale thing 



U: the most important thing is that you understand that 
the gate is just luck 



0: anything can happen, so that's why I think it's fair 

Figure 21: Luck means you can’t make any 
predictions 

In attempting to apply these findings on 
randomness to the creation of learning 
environments, we can see that students do have 
productive and normative ideas about randomness. 
Specifically, flipping coins is understood to be a 
non-deterministic event, and many of the students 
explicitly stated that for such an event variation is 



to be expected from trial to trial. However, some 
students also believe that such a non-deterministic 
event means that nothing at all can be predicted 
about future events, even to the point of excluding 
data (as will be discussed in the next section on 
data validity). It is important to note, though, that 
this can be viewed as a normative understanding of 
randomness that has been over-extended. That is, 
a frequentist understanding of probability does 
maintain that it is meaningless to apply probabilities 
to predict the outcome of specific events, so these 
students are in many ways “correct” when the 
apply this intuition to short-run data. It is only 
when extending this idea to long-run data that this 
reasoning becomes non-normative. 

The Outcome Space and Probability Distribution: 

In formal probability theory, when determining 
the probability of an event, one first determines the 
relevant outcomes (in the games discussed in this 
study, these are all the possible combinations of 
two or three coins). Then, creating or using a 
probability distribution, which assigns probabilities 
to each outcome in the outcome space, one 
determines the likelihood of each of the outcomes, 
and then combines probabilities to determine the 
probability of specific events (such as Team A 
scoring a point). Using such a theoretical model, 
one can clearly differentiate between the outcome 
space and the probability distribution, and indeed, 
these two entities are often introduced at different 
times in probability and statistics textbooks (e.g. 
Pitman, 1993). 

Although the students in this study did invoke 
ideas similar to the outcome space and the 
probability distribution in this study, they often 
reasoned in a way that made it difficult to 
distinguish between the two, especially when 
discussing the three-coin game. A working 
hypothesis is that it was easy for students to 
understand and verbalize the outcome space for the 
Two-Penny game, whereas the outcome space of 
the Three-Coin game was more complex, making it 
harder to discuss. As a result, even though the 
outcome space was always on the screen for the 
both games, students who referred to the outcome 
space when making predictions about the fairness 
of the Two-Penny game did not refer to the 
outcome space making predictions about the 
fairness of the Three-Coin Game (figure 22). 
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TUo Penny Game: 

0: because it's heads and tails. 

F: and it's two heads and two tails 

Three Coin Game: 

F: You will get more because you start cut first 

Figure 22: Outcome space referenced in Two-Penny Game 

A further problem students had when discussing 
the outcome space of Three-Coin game was in 
differentiating the individual outcomes (such as 
head-tail-head) from the combination of all 
outcomes that could score a point for a team. So, 
for instance, when asked the probability of the 
specific outcomes occurring, many of the students 
would state that those outcomes that score a point 
for Team A were more probable, because they 
expected Team A to win. Although such 
statements can be interpreted as students simply not 
understanding the environment, it is more 
consistent with their other statements to posit that 
the outcome space is not a well-formed concept 
with these students, and they have a difficult time 
understanding how to relate the outcome space to a 
complex event such as the probability of a team 
scoring a point. 

When students did discuss the outcome space, 
some of the students explicitly stated that order did 
not matter when differentiating between outcomes 
(figure 23), resulting in a behavior that could easily 
be seen as representativeness. However, this 
version of representativeness is based on not fully 
understanding how to properly enumerate and 
partition the outcome space. This can be contrasted 
with students who don’t expect “patterns” in data 
(figure 24). The latter seems closer to the 
traditional definition of representativeness, and is 
based upon applying a non-normative probability 
distribution to a normative enumeration of the 
outcome space. 



M: ... see, like, there's three of HHT, and 3 of TIH, 
but ally cne of HHH and are of TIT, so it's hard to 

get HHH, it's easier to get HHT 

Figure 23: not differentiating outcomes based on order 



D: atm, I chose those I guess cause it's like too rruch 
of a pattern, like Tails tails tail uh. . .yeah, sesns 
like less of a chance for it to go cn that sane cne 
all three times, and like tails heads tails, like, in 
a pattern like that, and like hexyls 

Figure 24: representativeness based on a non-normative 
probability distribution 



At other times students did not make a clear 
distinction between the outcome space and the 
probability distribution. These students made 
statements that were not only ambiguous as to 
whether they were talking about the number of 
outcomes or the probability of certain outcomes, 
but they would often switch between talking about 
numbers of outcomes and probabilities of 
outcomes (figure 25). In fact, these students 
sometimes seemed to be reasoning normatively, 
discussing the different outcomes that score a point 
for each team (although without explicitly counting 
the number of paths), but were then surprised 
when they later discovered that the outcomes were 
not equally divided between Team A and Team B. 

N: I think the game isn't fair, because there is ix> way 
for team b to get their a*ai, it's like an advantage 
for team a because all they have bo cb is just get 2 
h eads or 1 head with the nickel. 

E: and it's easy to just get two heads 

N: I knew, it's easy to just get a head with the nickel, 
so I den' t think it's fair 



D: Just look at that and think about it. Look at how 
Tean A soores: HHH, HHT, HTH 

C: that's most of then right there 

D: it's easier to get all that 

Figure 25: ambiguous statements 

As discussed in the sections on randomness and 
data validity, one set of students who ignored the 
outcome space when determining the fairness of 
the games did so for very principled reasons: they 
believed that, because the games were luck, 
nothing could be predicted about the outcomes, and 
so the games had to be fair. These students rarely 
made reference to the outcome space or to the 
probability distribution, as they explicitly stated 
that these were irrelevant to the fairness of the 
games. 

Due to the short time scale of this study (each 
student used PIE for less than 2 hours), as well as 
the prototype quality of PIE, this study was not 
concerned with any gains made by the students in 
terms of probability concepts. However, student 
use of the outcome space is one area in which we 
observed a marked difference between students 
predictions for the Three-Coin game and 
conclusions for the Three-Coin game. In the 
predictions for the Three-Coin game only two pairs 
of students determined that the game was unfair, 
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and only one of these pairs explicitly used the 
number of outcomes in their reasoning. In 
contrast, after playing the simulation and creating 
their conclusions, seven of the eight pairs of 
students realized that the game was unfair, and all 
seven pairs determined that the number of 
outcomes was the deciding factor in the game being 
unfair. 

In summary, students exhibited many different 
ideas when discussing the outcome space and the 
probability distribution, making any generalizations 
difficult. We can say, however, that many 
students have difficulty in distinguishing between 
similar outcomes, and many students have 
difficulty in distinguishing between the outcome 
space and the probability distribution. 

The Validity of Evidence: 

It is in documenting people’s beliefs about the 
validity of data that the misconceptions literature in 
probability is least controversial: it is well known 
that people, including trained statisticians, often fall 
prey to the law of small numbers (Tversky and 
Kahneman, 1982). However, the data from this 
study presents a picture that is not as clear-cut as 
the existing literature would lead one to believe. 
Although many of the students did exhibit behavior 
consistent with the law of small numbers at some 
times, students also expected variability between 
different trials of a random process (as discussed 
previously in randomness), and the students who 
have previously been described as believing that 
nothing can be predicted about a random process 
did not fall prey to the law of small numbers: in 
fact, they explicitly denied the relevance of data that 
was in contradiction to their theories, not accepting 
the data as relevant until they had created a scenario 
that could fit the data, a behavior that is consistent 
with findings from the science education literature 
(Chinn and Brewer, 1993). 

These different behaviors can be roughly 
characterized as being either data-driven or theory- 
driven 4 . Data-driven behaviors were characterized 
either by students’ unwillingness to create a theory 



4 Note that different students may be considered data- or 
theory-driven at different times, as three pairs of students 
were characterized as data-driven and three as theory-driven, 
two were neither completely data- nor theory-driven; and 
students may switch between being data-driven and theory- 
driven, depending on such factors as if they have yet 
formulated a theory. 



in the absence of data, or by their willingness to 
give up their theory after only a small number of 
points had been played (typically 10 points or less) 
(see figure 26). Theory-driven behaviors were 
characterized by the students’ unwillingness to 
believe the data when it was in conflict with their 
theory. When students behaving in a theory-driven 
manner first saw that the data was inconsistent with 
their theory, they explicitly denied the relevance of 
the data (figure 27), decided that the computer is 
cheating, or just ignored the data and claimed the 
truth of the original predictions. Two pairs of 
these students finally abandoned their contention 
that the Three-Coin game was fair, but only after 
running many simulations, noting the unfair score, 
and finally realizing the importance of the number 
of outcomes that scored for each team. 



Reasoning when predicting for Three-Coin Game 
U: Because tried it. 

Reasoning when concluding for Three-Coin Game 

U: this is not right. We are kind of right on this cne, 
but vie did this point higher 

IN typed: BECAUSE WE’RE VffENG. 

Figure 26: data-driven discussions of evidence 



K: I think it might be unfair, because A is winning 

L: although in real life it would be fair 

K: ytu reset it 5 times, and every time A is always 
winning 

L: it is fair, but the cxnpiter is c±>eating 

Figure 27: theory-driven discussions of evidence (K is in 
the process of abandoning his theory, and L is maintaining 
his theory in the face of counter evidence) 

The research literature shows that many people 
fall prey to the law of small numbers, and this 
study was not an exception. However, this study 
did illustrate that the story may not be as simple as 
people always believing data that is based on a 
small sample. In particular, some of the subjects in 
this study did not believe the data when the data 
was in conflict with a theory that they had 
proposed. This suggests that perhaps there is an 
interaction between people’s expectations and the 
validity that they are willing to attribute to data. 
We posit that this has not been appreciated in the 
past due to the artificial nature of the tasks that 
subjects were given, whereas in our tasks students 
were engaged and felt ownership of their 
predictions and conclusions. 
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Summary 

As the results from this study show, students 
invoked many intuitions about probability when 
reasoning about the activities in PIE. And these 
intuitions do not seem to be easily characterized by 
standard misconceptions in probability. Instead, 
these intuitions may better be described by noting 
their similarities and dissimilarities to specific 
concepts in probability, notably randomness, the 
outcome space, the probability distribution, and the 
role of data in determining probabilities. 

From Theory into Action: Designing a 
Learning Environment 

Although the original intent of PIE, to have 
students confront their misconceptions, turned out 
to be ill-conceived, we do feel that valuable lessons 
were learned. For one, this study allowed us to 
create a framework of probabilistic reasoning that 
will inform future versions of PIE. Additionally, 
we saw that a collaborative inquiry environment 
can lead to students expressing many intuitions 
about probability. This is important in at least two 
ways. First, as researchers, we can use such 
environments as a way to help us understand 
student intuitions about probability. That is, as the 
students collaborate, their reasoning can be 
recorded and then later analyzed, giving us an 
understanding of how students reason 
probabilistically in situations that are perhaps more 
authentic then can be replicated in either a survey or 
in a one-on-one interview. Second, as educational 
designers, it is important that we help students 
build models from their existing ideas about 
probability. To do this we must have students 
make these ideas explicit, and realize the strength 
and weaknesses of these ideas. 

By developing a framework that looks at student 
understandings with an eye to the probabilist’s 
understanding of randomness, the outcome space, 
the probability distribution over the outcome space, 
and the role of data in analyzing expectations, we 
can create activities that bridge the gap between 
what students know before probability instruction, 
and what probability instruction aims to teach. The 
creation of such activities differs radically from the 
traditional approach to instruction in which one 
starts by teaching Kolmogorov’s axioms or the 
multiplication rule. Instead, axioms and rules are 
derived through the learning activities. To create 
such learning activities, we should be guided by 



theoretical frameworks that provide a coherent and 
consistent approach to instruction. 

As mentioned at the beginning of this paper, PIE 
is based on the premises that learning environments 
should situate students in activities they consider to 
be legitimate and authentic, and not ignore the 
contribution of social practices and representational 
resources (Gordin et al. 1994; White, 1993a, 
1993b). We cannot be so naive, however, to 
believe that by simply exposing the students to 
activities that highlight the importance the four 
previously mentioned features of probabilistic 
reasoning that they will learn to apply such 
reasoning in all appropriate instances. Students 
have formed many competing intuitions, and it is 
not easy to coax students into developing a 
systematic and coherent way of viewing 
probability. Instead, a principled approach is 
required in which activities are created that will lead 
students to construct a systematic and coherent 
view of probability. The principled approach 
recommended here is to have the students 
participate in a progression of activities, where 
each activity is interesting and highlights an 
important aspect of the domain to be studied, and 
each activity extends the findings from the previous 
activities (White, 1993a, 1993b). By incorporating 
such a series of activities, students are engaged in 
authentic activities from the beginning, they can 
create simple models that are consistent with some 
aspects of their intuitive reasoning, and, as they are 
systematically exposed to more complex situations, 
the models (and their intuitions) can be revised, 
and need not be completely overhauled. By 
internalizing these models students will expand the 
intuitions that are most appropriate for the domain, 
leading the students to a normative view of the 
domain of probability. 

This study can help to identify the progression of 
activities that should be designed to help students 
build from their ideas to normative probabilistic 
reasoning. One such series of activities will be 
touched on briefly here, and future research will be 
required to determine if such a series of activities 
can be useful in helping students to understand 
normative probability. 

Such a series of activities can start off with an 
extensive discussion of fairness, and the different 
ideas students may have about fairness. In such a 
discussion it is expected that students will easily 
converge on the normative definition of fair 
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meaning that each team is equally likely to win. 
Note that this idea should continue to be raised 
throughout the activities, in a manner similar to that 
proposed by a “spiral” curriculum. In the next 
activity students can be given a simple scenario in 
which they investigate the law of small and large 
numbers. By seeing the relationship between the 
random event of a coin flipping, the outcome of 
this simple event, and the accumulation of many 
instances of this event, students can collaboratively 
come to see that there will be much variability after 
a small number of flips, but regularity after many 
hundreds of flips. It should also be noted that each 
of these ideas, variability and regularity, are ideas 
that hqve.been expressed by the students in this 
study. This aspect of the model should be revisited 
throughout the activities. The next set of activities 
can highlight the role of the outcome space in 
determining if games are fair. In this set of 
activities the elementary events are linked to the 
compound events, which are linked to the 
cumulative effect of many hundreds of events. 
The first model the students will investigate is that 
of equally likely outcomes, allowing the students to 
derive a model that is based on counting strategies. 
The next model will be based on outcomes that are 
not equiprobable (such as spinners with unequal 
areas, which are not yet in this prototype version of 
PIE), allowing students to derive a model that is 
based on the multiplication rule. Although this is 
the extent of the activities that we expect to 
implement for a seventh grade probability unit, it 
seems fairly obvious how we can extend this 
model to introduce more complex concepts in 
probability such as conditional probability, and 
more complex probability distributions. 

Conclusion and need for further study 

Students have many ideas about probability, and 
these ideas are not adequately described by simply 
stating that students are using heuristics such as 
representativeness. Instead, students invoke a large 
number of intuitions about probability, and these 
intuitions can be seen to roughly correspond to the 
concepts of randomness, the outcome space, 
probability distribution, and the role of data. By 
viewing students’ probabilistic intuitions in this 
way we expect that, although many of the 
misconceptions found in the literature are adequate 
ways of describing the behavior of some students 
some of the time, students will exhibit great 
variation in behaviors, based on their 
understanding of these four related areas. In fact, 
this variation is exactly what is observed in this 



study as well as in the research literature. We feel 
that by viewing student ideas about probability as 
consisting of four interrelated sets of intuitions, we 
can come to a more thorough understanding of 
probabilistic reasoning. 

The question is then raised as how we can best 
harness such student intuitions to improve 
understanding of probability. It is posited that by 
creating inquiry-based learning environments we 
can help students to examine and expand their ideas 
in creating an understanding of probability that is 
consistent with normative probability theory, and 
future research will be required to determine if this 
is a productive method of teaching probability. 
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